Case-based Modeling with Qualitative Indices

One of the challenges in process control is prov id ing reliable control of poor ly understood systems. Before such a system can be cont ro l led we must f i rst be able to predict its future behavior-so that we know what control action is necessary. Th is paper presents two approaches to this predict ion task, both using qual i tat ive models augmented by records of histor ical system behavior. Our hypothesis is that qual i tat ive in format ion about a system is more easily available than quant i tat ive equations; moreover, the in format ion need not be complete or to ta l ly correct. We restructure the histor ical in format ion into a case-base suitable for the predict ion task, and use the qual i tative model to identi fy the attr ibutes to use as case-indices. The case-base then provides the quant i tat ive in format ion needed for the predict ion task. Our techniques are extensively evaluated on data taken f r om a real-world system. 1 I n t r o d u c t i o n Understanding physical systems well enough to predict and control their behavior has long been a goal in engineering and science. When systems are simple, numerical equations can exactly reproduce the system's behavior. For complex systems, however, developing an accurate numerical model is rarely feasible. Qual i ta t ive model ing alleviates this problem by modeling systems at a higher level of abstract ion. A qual i tat ive model seeks to ident i fy and model only the most impor t ant aspects of a system. In the classic example of a bal l th rown into the air , an abstract model w i l l not at tempt to predict how h igh the ba l l r ises-but only that i t w i l l rise to some height, reverse d i rect ion, and fa l l to the f loor. Qual i ta t ive model ing techniques such as QSIM [8] and qual i tat ive process theory [3] have been successfu l at model ing complex physical systems at this level. However, predict ions at the qual i tat ive level are insufficiently precise for tasks that do require numerical results, such as diagnosis and process contro l . Techniques such as Q3 [2] SODE [7] and SIMGEN allow combining the qual i tat ive models wi th numerical equations to obtain precise results. However, such techniques are applicable only when these numerical equations are known, again restr ict ing them to well-understood processes. For many practical systems, this is not the case. For example, the tests in this paper were carried out on a coffee roaster, used in various plants of Nestle, for which it has so far been impossible to construct an accurate numerical model. The absence of accurate numerical predict ions has led to many cr i t ical situations, such as fires, which destroy the entire load of coffee beans and require expensive shutdowns. Since prevention of these cr i t ical situations depends on the numerical predict ion of key parameters, purely qual i tat ive models are insufficient. However, an enormous amount of in format ion about the roaster is available in the fo rm of records of past behavior. An alternative to generating a numerical model is to use case-based reasoning, where predictions are based on previous experiences. Marc Goodman [5] has reported promis ing results using this paradigm to make predictions about the behavior of a complex video game. The major problem for such an approach is indexing: which of the wealth of previous observations are in fact relevant to the current si tuat ion? In this paper, we describe two ways in which a qual i tat ive model can be used to provide such indices, al lowing us to combine past experiences into a predict ion for the current s i tuat ion. A first approach to locat ing relevant precedents is nearest-neighbor search. Here, the problem is to f ind an appropriate s imi la r i t y metr ic which assigns weight to those aspects which are impor tan t for the predict ion. We have implemented a system where this metr ic is determined based on a qual i tat ive model , and tests on actual coffee roaster data have shown satisfactory results. In another approach, first mentioned by Hellerstein [6], the qual i tat ive model is used to determine which experiences can provide bounds on the current s i tuat ion. We have also implemented this second approach and have obtained very promis ing results. RICHARDS, FALTINGS, AND DUXBURY-SMITH 1757 When predict ion concerns a commonly occurr ing s i tuat ion, it is usually possible to locate a single almost identical past experience that gives the correct predict ion . However process predict ion is most impor tan t in cr i t ica l si tuations for which past experience is (hopeful ly) sparse. A predict ion may then have to be construed f r om several precedents, none of which is entirely s imi lar to the current s i tuat ion. More precisely, each parameter may be predicted f r om cases selected according to cr i ter ia specific to that parameter. Our approach is capable of intel l igent ly combining in format ion f r om several cases in to a single solut ion. Using records of histor ical behavior to provide quanti tat ive in format ion has the marked benefit that the info rmat ion is " f ree". In other words, where development of a numeric model requires a great deal of effort, developing a l ib rary of cases requires only that we moni tor the system and record what it does-something that is a normal par t of most process-control systems. Our techniques offer two further benefits. F i rst , when dealing w i th complex machines, each machine may well have its own ind iv idual characteristics w i th in some range of "no rma l " behavior. By using cases recorded f rom each machine, we automatical ly account for these ind i v idual variat ions. Second, although our techniques take as input qual i tat ive in format ion about the system, they do not require this in format ion to be complete or correct; accuracy degrades gracefully as the qual i tat ive model deteriorates.